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ABSTRACT 

The education-production function has become the 
dominant paradigm for analyzing the effects of education resources on 
student outcomes. This paper presents findings of a study that 
investigated the education-production function and its relationship 
to educational policymaking. Fiscal data and student-achievement test 
scores for Indiana during the 1993-94 school year were analyzed using 
Fortune and O'Neil's (1994) methodology as an alternative to 
traditional production-function analysis. The model uses t-tests for 
determining significance and comparing homogeneous subgroups. 
Although the alternative methodology is more likely to reveal 
relationships between inputs and outcomes, the Indiana data revealed 
no significant relationships between expenditures and student 
achievement. However, when analysis of socioeconomic status and 
student achievement was conducted, there was a high correlation 
between these variables. Fortune and O'Neil concluded that the only 
constitutionally relevant outcome of education is whether or not 
students receive equal access to state resources. This study 
demonstrates that all students in Indiana do not have equal access to 
resources. One problem with production-function analysis is that it 
assumes that all schools pursue the same goals and that the goals are 
related to student achievement. Second, production-function analysis 
inadequately identifies inputs and ignores processes. For example, 
production-function analysis overlooks curriculum and instruction as 
an input. A conclusion is that increased revenues must be used to 
find new strategies for improving student achievement. Five tables 
are included. (Contains 14 references.) (LMI) 
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Abstract 

This study investigates the education production function and discusses 
its relationship to educational policymaking. Fiscal data and student 
achievement test scores for Indiana during the 1993-94 school year are used in 
the production function analysis. The research methodology is different than 
traditional production function analysis; an alternative methodology proposed by 
Fortune & O'Neil (1994) using t-tests for significance and comparing 
homogeneous subgroups was adopted for this study. Although the alternative 
methodology is more likely to reveal relationships between inputs and 
outcomes, the Indiana data revealed no significant relationship between 
expenditures and student achievement. However, when analysis of 
socioeconomic status and student achievement was conducted there was a high 
correlation between these variables. The utility of education production 
function analyses in policy formulation and decision making is considered, 
especially in light of the results of the Indiana investigation. 
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An Indiana Investigation of the Impact of Expenditures & Socioeconomic 
Status on Student Achievement: Does Money Matter? 
Introduction and Background 

In the wake of the national educational reform bill, Goals 2000: Educate 
America Act, the question of whether allocating and spending more money on 
education will actually increase student productivity arises in policy discussions. 
Odden (1994) indicates that even before the recent focus on higher student 
achievement, the realization that funding has been substantially increasing while 
student achievement has remained flat has led some to the conclusion that 
education has a productivity problem. The prospect of spending more money 
on education nationally and within states generates controversy because 
researchers have contrary findings (e.g. Hanushek, 1989; Monk, 1989; Baker, 
1991; Card & Krueger, 1992; Wainer, 1993; and Hedges, Laine, & Greenwald, 
1994a, 1994b) concerning whether differential school inputs actually influence 
student outcomes. The more technical term for this type of research is 
education production function analysis. 

Over the last decade the education production function has become the 
dominant paradigm used to analyze the effects of education resources on student 
outcomes (Greenwald, Hedges, & Laine, 1994). Production function studies 
attempt to develop a model of the relation between educational inputs and 
outcomes. Greenwald, Hedges, & Laine define inputs to include school 
resources such as expenditures, teacher characteristics, or facilities, and student 
characteristics such as socioeconomic status or ability. They define outcomes as 
achievement as measured by standardized tests, future educational patterns, and 
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adult earnings. Furthermore, the goal of production function studies is to 
develop quantitative models that can predict the effect on student outcomes of a 
given change in inputs. Therefore, in theory a production function could 
predict how much the median achievement on a standardized test would change 
if for example, per-pupil expenditures were increased by $500 (Greenwald, 
Hedges, & Laine). They also point out that although there is no consensus on 
the exact specification of the educational production function, there are broad 
guidelines regarding the specification of models, and more specifically, on 
measures of school resources, student characteristics, and outcomes. 

Although education production function is considered a viable research 
method for analyzing resource inputs and educational outcomes, there is 
controversy concerning its methodology and the policy relevance of its results. 
For example, in a meta-analysis of 187 studies using production function 
methodology Hanushek (1989) concluded that there is no systematic relationship 
between resource inputs and school outcomes when controlling for students 
characteristics and socioeconomic status. Hedges, Laine & Greenwald (1994a) 
contend that the method Hanushek employed in his meta-analysis, vote- 
counting, is weak as an inference procedure and has low statistical power. 
Instead, their meta-analysis of the same studies using more sophisticated 
synthesis methods (i.e. combined significance tests and estimation methods) 
shows systematic positive relations between resource inputs and school 
outcomes. Hanushek (1994) responds to Hedges et al. (1994a) by defending 
his previous conclusions and arguing that "policy makers should not be confused 
into believing that throwing money at schools in effective" (p. 8). In response, 
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Hedges et al. (1994b) defend their research methodology and support their 
conclusions that there is a positive relationship between resources and school 
outputs and contend tha the focus should now shift to how money matters. 

Fortune & O'Neil (1994) also illuminate problems with the production 
function approach and Hanushek's (1989) conclusions and contend that 
correlational methods for analysis may hide rather than reveal relationships. 
They propose the use of t-tests for significance as an alternative method for 
analysis of expenditure and achievement data. For example, by examining 
homogeneous sets of school districts based on the key dimension of wealth and 
conducting several t-tests for significance, possible relationships between 
expenditures and achievement that were overlooked by the traditional production 
function method are exposed. 

Considering the controversy surrounding traditional education production 
function research described above, the research we conducted attempts to 
replicate the methodology proposed by Fortune & O'Neil (1994;. Their 
alternative production function methodology is applicable if the policy question 
of interest is whether differences in funding relate to current differences in test 
scores. Since educational funding equity and revision of Indiana's school 
funding formula are prevailing policy issues we decided to use the alternative 
production function methodology to investigate whether differences in funding 
are related to differences in test scores. We use achievement test data for the 
state Indiana (Indiana Statewide Educational Testing Program - ISTEP) and 
compare it with school district per pupil expenditures to determine if significant 
differences in achievement exists between high spending and low spending 



Education Production Function 



6 



districts. We also expanded our investigation and examined socioeconomic 
status (SES) as a student input characteristic (Greenwald, Hedges & Laine, 
1994) and compared it with district ISTEP scores. Further, we examine the 
limitations of our research and discuss the implications of our findings for 
policy makers using production function research to formulate educational 
policy. 

Methodology 

Fortune and O'Neil (1994) levy an attack on the appropriateness of 
production function research to assess educational funding equity. Their 
critique centers around methodological limitations, including the incorrect use 
of correlational methods, attenuated variables, and shared variance issues 
associated with multiple regression techniques. They also point out that the 
methodological problems limit its usefulness for policy research. In order to 
make the production function approach policy relevant, they alter the production 
function analysis in three ways. First, they contend that the task should be a 
comparison rather than an association. Second, rather than looking for a 
consistent relationship across the whole population, they believe it is better to 
ask for what kinds of districts such effects exist within a state. The third 
alteration is to create a discrepancy in expenditures large enough to expect 
differences in the purchasing power of educational services. 

Since a multitude of problems exist with the traditional method of 
production function analysis, we adopted Fortune and O'Neil's (1994) 
alternative production function methodology to study the educational funding 
equity in public schools in Indiana. More specifically, we have investigated 
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"the only question truly addressed by productio: unction studies:" (Fortune & 
O'Neil, 1994, p. 22) Are differences in per pupil expenditures associated with 
differences in achievement test scores? We analyzed test score data from 
Indiana's elementary schools and general fund expenditure data from the 
corresponding school districts. The test score data is from Indiana's Statewide 
Test of Educational Progress (ISTEP), a test which is taken each year in Indiana 
by students in grades two, three, six, and eight. Mandatory summer 
remediation classes (known as Extended Learning Program) are required for 
students whose ISTEP scores fall below a statewide cutoff score (Indiana 
Education Policy Center, 1994). 

Statistical data was obtained from the Indiana State Department of 
Education by downloading it from Ideanet, the Department of Education's 
computer system (Internet node) designed to provide a variety of educational 
data to school systems and universities within Indiana. The following data were 
collected for all public schools within the state: SES(socioeconomic status), 
ISTEP total battery (Indiana Statewide Test of Educational Progress), ISTEP 
language proficiency, ISTEP mathematics proficiency, district general fund 
expenditures, and ADM (Average Daily Membership). All data were for the 
1993-94 school year. 

All elementary schools were grouped by district. In order to create 
homogeneous subgroups for analysis, outliers within the sample were deleted 
from the data set. These outliers included (a) all schools which were designated 
as strictly special education centers, (b) all schools designated as vocational 
education centers, (c) all school districts in excess of 25,000 students, and (d) 



Education Production Function 



8 



all school districts with fewer than 300 students. Additionally, a district with an 
extremely low general fund expenditure was deleted, due to our belief that the 
data point was clearly erroneous in comparison to the other data. 

Per pupil expenditures were calculated by taking the general fund 
expenditures (the only expenditure category available on the database) reported 
for the district and dividing by the district's reported Average Daily 
Membership (ADM). This quotient is the value reported for per pupil 
expenditures throughout this study. According to Fortune & O'Neil (1994), the 
data should be ordered by instructional expenditures per pupil and then divided 
into two equal groups -the upper and lower 30 percent of the sample. They also 
require that a disparity of at least $700 between districts be established. This 
figure is based on the amount of money (prorated) that a 1970 study established 
as what was needed to improve elementary school student's reading scores by a 
one-month-of-training-experience level over the course of a year. In our data 
set a funding threshold of $893.86 was established; the difference between the 
average of the upper 30% of the. elementary sample and the average of the 
lower 30% of the elementary sample. This value clearly exceeds the $700 
disparity between homogeneous subgroups suggested by Fortune and O'Neil. 
The per pupil expenditure data was calculated on a per district basis, which 
meant that ISTEP scores had to be averaged for all the elementary, middle, and 
high schools in that district. It was this average that was used as the district 
ISTEP score. The ISTEP scores are T-scores. 

Independent t-tests with pooled variance were then used to check for 
significant differences between ISTEP test scores between the previously 
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mentioned upper and lower samples. The means, variance, number of schools, 
districts, P values and significance at the a = .05 level are reported in the tables 
that follow in the next section. 

Results 

The results of this study indicate that there is no significant difference in 
elementary student achievement scores (as measured by the ISTEP) based on per 
pupil expenditures. Using the upper and lower 30% of the educational 
expenditures to establish the two groups, a funding threshold of $893.86 
separates the average expenditure of the groups. Fortune and O'Neil (1994) 
suggest that this should be, within a single homogeneous subgroup, capable of 
demonstrating differences in achievement scores. We did not find this to be the 
case with our sample. 

Our results are displayed in Table 1. A total of 170 districts make up 
the upper and lower 30% of the total elementary sample. Comparison of the 85 
districts in the lower 30% of the sample to the 85 districts in the upper 30% of 
the sample demonstrates no significant difference in mean ISTEP test scores on 
either the total battery, the language proficiency, or the mathematics proficiency 
sections. In fact, in all cases, the higher expenditure districts actually had 
slightly lower test scores and much greater variation among scores than the 
lower expenditure districts. This result may be due in part to the fact that 
general fund expenditures include the state basic grant revenue for special 
education, vocational education, at-risk, and categorical grants. Also included 
in the general fund are other state grants for summer school, adult education, 
gifted and talented, ISTEP, performance based awards, textbook 
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reimbursement, social security, and PRIMETIME. Capital outlay and debt 
service are not included in general fund revenues. Therefore, it is possible that 
the "mix" of grants for "at-risk students" included in general fund expenditures 
creates districts with high expenditures that have a "mix" of student abilities, 
many of whom are low achievers. This could explain the lower mean scores 
and greater variance in scores than their lower expenditure counterparts. 

[Insert Table 1 about here] 
Believing that perhaps an inadequate funding threshold had been 
established, we conducted t-tests for the upper and lower 15% of the sample. 
As shown in Table 2, this creates a funding threshold of $1304.75. The results 
of the t-tests for those 84 districts still demonstrated no significant difference 
between the two groups. 

[Insert Table 2 about here] 

When simple Pearson correlations were run for the entire data set, it was 
determined that the highest correlations existed between SES and ISTEP scores. 
In fact, the correlations were on the order of 0.73 on average. Other 
interesting, non-zero, correlations exist between SES and attendance (r=0.72) 
as well as between attendance and ISTEP scores (r=0.60). Since 
socioeconomic status (SES) is a student characteristic that is considered an input 
measure (Green wald, Hedges, & Laine, 1994) an analysis of ISTEP scores 
against SES data was undertaken. Table 3 demonstrates the results which were 
obtained using this comparison. 

[Insert Table 3 about here] 



Education Production Function 



11 



The data for these t-tests were obtained in a similar manner to the 
recommendations of Fortune & O'Neil (1994). All elementary schools were 
ordered by increasing socioeconomic status (SES) values. SES, as defined by 
the Indiana Department of Education, is the percentage of students not on free 
or reduced lunch programs. An SES of 0.950 means that 5 percent of the 
students are on free or reduced lunch programs at that school. Once the schools 
were ordered by increasing SES, two groups were established by taking the 
upper and lower 30% of the total sample now based on SES. 

The results indicate that the high SES school districts scored significantly 
better than the lower 30% group on the three reported achievement measures of 
the ISTEP test (battery, language, math). The large n in this sample (n=347) is 
due to the fact that SES was reported for each school along with the schools 1 
individual ISTEP scores. Therefore, no across-district averaging was necessary 
as it was when using PPE as the input variable. 

The results of this comparison demonstrate the significant differences 
between high and low SES districts. The overall total battery on the ISTEP 
shows a difference of nearly 12 points between the upper and lower groups 
based on SES. The language and mathematics subtests showed approximately a 
17 and 16 point difference respectively. One need not even bother with t-tests 
to determine that significant difference exists between the two groups. 

Since elementary schools revealed such an obvious difference between 
upper and lower groups, it was decided that middle school and high school data 
should also be evaluated using this methodology. The middle school data is 
shown in Table 4. The same variables as the elementary data are evaluated. 
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The high school data is shown in Table 5, and includes additional comparisons 
to SAT data and graduation rate data for low and high SES student populations. 

[Insert Tables 4 & 5 about here.] 

Again, both the middle school and high school data suggest that large 
differences exist between the lower and upper 30% of each sample based on 
socioeconomic status (SES). Effectively a 19 point difference exists between 
the lower and upper groups on the language proficiency portion of the ISTEP 
for the middle school students. A 12 point difference exists for the lower and 
upper groups on language proficiency for the high school students. Similar 
results are obtained for the total ISTEP battery and math proficiency sections, 
with the higher SES group consistently outscoring, by a substantial margin, 
their lower SES counterparts. 

When we focus on the additional high school independent variables of 
SAT scores and graduation rates, we find similar results. The SAT scores of 
the high SES group are on average 54 points higher than the low SES group. 
While the difference in points is perhaps not remarkable, it should be kept in 
mind that the sample of students taking the SAT is significantly reduced from 
the sample of students who are required to take the ISTEP. The group electing 
to take the SAT is much more homogeneous than the population which is 
required to take ISTEP. 

The variation in graduation rates at the high school level is cause for 
concern. According to our data, the lowest 30% of our students based on SES 
have a 10% lower graduation rate than the highest 30%. In a democracy in 
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which we purport to offer equal educational opportunities to all students, we are 
clearly falling short of that mark. 

Limitations 

This analysis is plagued by many of the errors inherent in production 
function analyses. First, the study is based on aggregated data. The per pupil 
expenditures are determined at the district level, not by each individual school 
or program. Therefore, it is impossible to determine if certain schools within a 
given school district receive differential per pupil expenditures. Also, the 
availability of general fund expenditures in the data set, rather than instructional 
expenditures limits the accuracy of the per pupil expenditure figures. Likewise, 
the ISTEP scores are also aggregated data. In districts with multiple elementary 
schools, the ISTEP scores for all schools were averaged, and the mean score 
reported as the district ISTEP score for the battery, language, and/or math 
proficiency. Monk (1992) recommends that future analyses focus on the 
classroom as the unit of analysis. Determining per pupil expenditures, 
socioeconomic status and other independent variables on a classroom by 
classroom basis and comparing it with educational outputs from the same 
classrooms is a potentially valuable research endeavor. 

In this study, socioeconomic status is clearly a variable which is 
impacted by many other intervening variables. Socioeconomic status includes a 
variety of other variables which are not separated out in this analysis, such as 
stability of the home, educational level of the parent(s), or percentage of adults 
below the poverty line in the district among others. Clearly, shared variance is 
a problem in this analysis, and effects the true relationship between the inputs 



Education Production Function 



14 



and outputs, as it does in all production function studies of this nature (Fortune 
& O'Neil, 1994). 

Discussion and Conclusions 

A primary responsibility of policy makers is to ensure efficient use of ' 
taxpayer dollars. Consequently, when research reveals that spending more 
money on education does not yield significant gains in student achievement, 
policy makers begin to seriously question or completely reject requests for 
additional funding of K-12 schools. Sadly, even using the alternative 
production function methodology proposed by Fortune & O'Neil (1994) the 
expenditure and achievement test data for Indiana yields this result. Our 
nonsignificant results demonstrate Fortune & O'Neil's conjecture that 
production function analysis is "inappropriate for the determination of school 
funding equality because it attempts to link inputs with one or a few particular 
output(s) of the education process such as test scores" (p. 22). Furthermore, 
"the production function methodology ignores and detracts from the 
fundamental issue of equal access by students to the resources of the state 
system of education" (Fortune & O'Neil, p. 22). They conclude that the only 
constitutionally relevant outcome of education is whether or not students receive 
equal access to the resources of the state. Clearly, our analysis using 
socioeconomic status (SES) as an input variable and student achievement as the 
outcome variable demonstrates that all students in Indiana do not have equal 
access to resources. 

Production function research is widely shared with policy makers as 
evidence that spending more money on education does not increase student 
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achievement. However, policy makers must be made aware that there are 
several reasons why this type of research has been relatively unsuccessful in 
identifying relationships between resources inputs and student achievement that 
are not methodological in nature. Odden & Picus (1992) identify several 
problems with production function analysis. First, the assumption is made that 
all schools are pursuing the same goals and that the goals are related to student 
achievement. This may not be true. Schools pursue many goals and student 
achievement may not be the primary one. Also, they point out that standardized 
achievement tests are not necessarily indicators of what students have learned in 
school. 

Another problem Odden & Picus (1992) associate with production 
function research is the fact that it is difficult to identify inputs. There is no 
consensus on the types of input to analyze. In addition, since production 
function studies attempt to relate inputs to outcomes, they ignore processes. A 
critical process variable that is overlooked is curriculum and instruction which is 
certainly a significant factor that is linked to student learning. With this fact in 
mind, one of the five characteristics in Clune's (1993) conceptual framework 
for systemic educational policy is regular assessment of educational inputs, 
outcomes, and processes. Accordingly, student assessment in this model would 
correspond to curriculum goals. Hence, the current movement toward systemic 
educational reform would certainly limit the utility of production function 
research in evaluating how resources influence student achievement. 

In conclusion, policy makers should exercise caution when evaluating 
educational productivity based on production function research. Odden & Picus 



ERIC 



Education Production Function 16 



(1992) conclude that the message of production function research is not that 
money does not matter, but rather, "...that if additional education revenues are 
spent in the same way as current education revenues, student performance 
increases are unlikely to emerge" (p. 281). Hence, increased revenues must be 
used to find new strategies to improve student achievement. In the current wave 
of educational reform, policy makers must determine if increasing student 
achievement is the ultimate and only outcome they are hoping to accomplish. 
Perhaps there are other important goals and outcomes that education should seek 
to instill in students that education production function cannot measure and 
assess. The task of policy makers then is to design coherent educational policy 
and implement systemic change in education that will improve rather than 
impede the system. A daunting task, but one that is already underway. 
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Table 1 

U pper and Lower 30% by Per Pupil Expenditures" 
Contrasts of High and Low Funded Districts 



ISTEP Test Mean Variance n t Significance 



Total Battery 0.481 n.s. 

--High 65.10 31.13 85 

--Low 65.44 13.46 85 

Language Proficiency 0.645 n.s. 

-High 50.67 52.29 85 

--Low 51.28 24.00 85 

Math Proficiency 1.015 n.s. 

-High 50.99 60.09 85 

-Low 52.02 26.59 85 



* Indiana Per Pupil Expenditures $3528.99 to $4422.85 (n=292) 
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Table 2 

U pper and Lower 15% by Per Pupil Expenditures * 
Contrasts of High and Low Funded Districts 



ISTEPTest Mean Variance n t Significance 



Total Battery 0.638 n.s. 

-High 64.75 33.41 42 

--Low 65.44 15.10 42 

Language Proficiency 0.0.913 n.s. 

-High 50.00 60.06 42 

-Low 51.32 27.88 42 

Math Proficiency 1.006 n.s. 

-High 50.66 64.73 42 

-Low 52.15 27.09 42 



* Indiana Per Pupil Expenditures $3427.94 to $4732.69 (n=292) 
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Table 3 

U pper and Lower 30% by SES Contrasts of High and Low SES Elementary 
Districts '" 



ISTEP Test Mean Variance n t Significance 



Total Battery -26.8 7.0E-106 

--High 69.44 21.82 347 

--Low 57.75 44.28 347 

Reading Proficiency -26.6 1.57E-99 

-High 56.36 30.50 347 

-Low 39.54 109.0 347 

Math Proficiency -24.9 2.1E-93 

-High 56.58 37.40 347 

-Low 40.58 102.2 347 



* (n = 1156) 
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Table 4 

U pper and Lower 30% bv SES 

Contrasts of High and Low SES Middle School Districts '" 



ISTEPTest Mean Variance n t Significance 



Total Battery -16.3 1.49E-34 

--High 64.64 14.53 84 

--Low 52.14 34.86 84 

Reading Proficiency -17.2 5.72E-36 

-High 57.93 25.08 84 

-Low 39.44 72.25 84 

Math Proficiency -15.7 3.94E-32 

-High 57.54 26.64 84 

-Low 39.69 82.29 84 



* (n=280) 
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Table 5 

U pper and Lower 30% by SES 

Contrasts of High and Low SES High School Districts * 

ISTEP Test Mean Variance n t Significance 



Total Battery -12.2 1.39E-25 

--High 64.04 15.17 102 

--Low 55.79 31.50 102 

Reading Proficiency -11.3 1.44E-22 

-High 54.45 33.70 102 

-Low 42.09 89.07 102 

Math Proficiency -11.9 1 . 79E-24 

--High 55.22 37.58 102 

-Low 41.88 90.74 102 

SAT -6.23 1.69E-09 

-High 889.95 2365 102 

-Low 835.59 5403 102 

GRAD RATE -7.89 3.25E-13 

-High 0.89 0.003 102 

-Low 0.79 0.013 102 



(n=340) 
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